fix: splunk output plugin correct record accessor key for hec_token #8793
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mannbiher
requested review from
edsiper,
leonardo-albertovich,
fujimotos and
koleini
as code owners
Signed-off-by: Maneesh Singh <[email protected]>
mannbiher
force-pushed
the
splunk_out_plugin_incorrect_auth
branch
2 times, most recently
from
efdb995 to
e16c2e0
Compare
cosmo0920
approved these changes
May 10, 2024
|
thanks for your contribution. Note that commits must be prefixed wi the interface name according to https://github.com/fluent/fluent-bit/blob/master/CONTRIBUTING.md#commit-changes |
edsiper
added a commit
that referenced
this pull request
May 23, 2024
The following patch perform 2 changes in the code that helps to fix the problems found with Splunk hec token handling: 1. In the recent PR #8793, when using the record accessor API flb_ra_translate_check() to validate if the hec_token field exists, leads to noisy log messages since that function warns the issue if the field is not found. Most of users are not using hec_token set by Splunk input plugin, so their logging gets noisy. This patch replaces that call with flb_ra_translate() which fixes the problem. 2. If hec_token was set in the record metadata, it was being store in the main context of the plugin, however the flush callbacks that formats and deliver the data runs in separate/parallel threads that could lead to a race condition if more than onen thread tries to manipulate the value. This patch adds protection to the context value so it becomes thread safe. Signed-off-by: Eduardo Silva <[email protected]>
edsiper
added a commit
that referenced
this pull request
May 23, 2024
The following patch perform 2 changes in the code that helps to fix the problems found with Splunk hec token handling: 1. In the recent PR #8793, when using the record accessor API flb_ra_translate_check() to validate if the hec_token field exists, leads to noisy log messages since that function warns the issue if the field is not found. Most of users are not using hec_token set by Splunk input plugin, so their logging gets noisy. This patch replaces that call with flb_ra_translate() which fixes the problem. 2. If hec_token was set in the record metadata, it was being store in the main context of the plugin, however the flush callbacks that formats and deliver the data runs in separate/parallel threads that could lead to a race condition if more than onen thread tries to manipulate the value. This patch adds protection to the context value so it becomes thread safe. Signed-off-by: Eduardo Silva <[email protected]>
markuman
pushed a commit
to markuman/fluent-bit
that referenced
this pull request
May 29, 2024
Signed-off-by: Maneesh Singh <[email protected]> Signed-off-by: Markus Bergholz <[email protected]>
markuman
pushed a commit
to markuman/fluent-bit
that referenced
this pull request
May 29, 2024
The following patch perform 2 changes in the code that helps to fix the problems found with Splunk hec token handling: 1. In the recent PR fluent#8793, when using the record accessor API flb_ra_translate_check() to validate if the hec_token field exists, leads to noisy log messages since that function warns the issue if the field is not found. Most of users are not using hec_token set by Splunk input plugin, so their logging gets noisy. This patch replaces that call with flb_ra_translate() which fixes the problem. 2. If hec_token was set in the record metadata, it was being store in the main context of the plugin, however the flush callbacks that formats and deliver the data runs in separate/parallel threads that could lead to a race condition if more than onen thread tries to manipulate the value. This patch adds protection to the context value so it becomes thread safe. Signed-off-by: Eduardo Silva <[email protected]> Signed-off-by: Markus Bergholz <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In Splunk output plugin fix record accessor key for hec_token. Record accessor key should start with
$elseflb_ra_translatefunction simply returns the key as value.flb_ra_translatereturns an empty string if key is not found which would passif (hec_token)check, so we need to useflb_ra_translate_checkfunction to return NULL if key is not found.At this function
fluent-bit/plugins/out_splunk/splunk.c
Lines 350 to 369 in 3529bbb
if (ctx->event_sourcetype_key)existence before searching forhec_tokenkey. Either we should check formetadata_auth_keyin which case if statement is redundant asmetadata_auth_keyis always defined.fluent-bit/plugins/out_splunk/splunk_conf.c
Line 266 in 3529bbb
If it is meant to be
event_keyas the original pull request shows in example, I can change it and commit.#8738
Fixes #8781
Enter
[N/A]in the box, if an item is not applicable to your change.Testing
Before we can approve your change; please submit the following in a comment:
logs from splunk output plugin process
logs from fluent-bit splunk input plugin process. It shows that splunk token is respected and not overridden.
Valgrind fluent-bit splunk input plugin
Valgrind fluent-bit splunk output plugin
If this is a change to packaging of containers or native binaries then please confirm it work.
ok-package-testlabel to test for all targets (requires maintainer to do).Documentation
This is a bug fix
Backporting
Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.